Fast Adaptation with Bradley-Terry Preference Models in Text-To-Image Classification and Generation

Recently, large multimodal models, such as CLIP and Stable Diffusion have experimented tremendous successes in both foundations and applications. However, as these models increase in parameter size and computational requirements, it becomes more challenging for users to personalize them for specific tasks or preferences. In this work, we address the problem of adapting the previous models towards sets of particular human preferences, aligning the retrieved or generated images with the preferences of the user. We leverage the Bradley-Terry preference model to develop a fast adaptation method that efficiently fine-tunes the original model, with few examples and with minimal computing resources. Extensive evidence of the capabilities of this framework is provided through experiments in different domains related to multimodal text and image understanding, including preference prediction as a reward model, and generation tasks.Comment: Submitted to EYSM23 proceeding

ZYN: Zero-Shot Reward Models with Yes-No Questions

In this work, we address the problem of directing the text generations of a LLM towards a desired behavior, aligning the generated text with the preferences of the human operator. We propose using another language model as a critic, reward model in a zero-shot way thanks to the prompt of a Yes-No question that represents the user preferences, without requiring further labeled data. This zero-shot reward model provides the learning signal to further fine-tune the base LLM using reinforcement learning, as in RLAIF; yet our approach is also compatible in other contexts such as quality-diversity search. Extensive evidence of the capabilities of the proposed ZYN framework is provided through experiments in different domains related to text generation, including detoxification; optimizing sentiment of movie reviews, or any other attribute; steering the opinion about a particular topic the model may have; and personalizing prompt generators for text-to-image tasks. Code to be released at \url{https://github.com/vicgalle/zero-shot-reward-models/}

Sperm physiology and quality in two marine teleosts: Anguilla anguilla & Takifugu niphobles

The conservation status of the species studied in this thesis (European eel and pufferfish) is currently frail, thus the main goal of this research was to develop, improve and apply several techniques and protocols with the aim of increasing the knowledge about their sperm biology, improving their reproductive performance and even helping future breeding in captivity. The reproductive performance of the males is often assessed through the sperm motion parameters analysed by the CASA system, so first we focused on how to standardize this technique in terms of procedural and biological settings. In this respect, we laid the foundations for applying a standard method to assess sperm quality in fish, making it possible for sperm studies to be compared both intra- and inter-laboratories using the proper CASA settings. Secondly, with the aim of improving the reproductive performance of European eel males, 3 thermal regimes (two of them variable: T10 and T15; and one of them constant: T20) and 3 hormonal treatments (hCG, hCGrec and PSMG) were assessed based on different sperm quality parameters. In the case of the thermal regimes, our results demonstrated that the onset and progression of spermiation are strongly influenced by water temperature, with treatment T20 showing the best results in all the sperm quality parameters. In the case of hormonal treatments, hCGrec produced the best results in all the sperm quality parameters, becoming an economically profitable/viable treatment and an effective alternative to the standard hCG treatment often used to induce spermiation in eel species. A preliminary physiological study regarding the changes to the main ions involved in the fish sperm activation process was carried out. Our results showed that intracellular concentrations of Ca2+ and K+ increased upon eel sperm activation, while pH gradually decreased over time, thus it is likely that all of them play an important role in the initiation of sperm motility in the European eel, as with other marine and freshwater teleosts. In the second part of this thesis, which focuses on the pufferfish, an in-depth study into the sperm of this species was carried out for future application in aquacultural matters. A short-term storage method for pufferfish sperm was developed, enabling us to preserve the sperm quality parameters for a relatively long time period (7 days) compared to fresh sperm samples. Moreover, the effects of both the osmolality and the ion composition of the activation media on the sperm motion parameters were evaluated, concluding that both factors play an essential role in the initiation of sperm motility of pufferfish sperm and probably, in marine fish sperm. Finally, in vitro fertilization trials were developed to assertain how the quantity and quality of male gametes affects the fertilization and hatching rates. We demonstrated that sperm/egg ratio and sperm quality are strongly related factors, suggesting that both should be taken into account as unique interrelated elements. In addition, coefficients of correlation among all the spermatozoa motion parameters provided by a CASA system and fertilization and hatching rates were estimated for the first time in a marine species. Spermatozoa velocities showed the highest coefficients (¿0.80), suggesting that the kinetics of the spermatozoa are a key factor in the fertilization process.Gallego Albiach, V. (2013). Sperm physiology and quality in two marine teleosts: Anguilla anguilla & Takifugu niphobles [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/34625TESISPremiad

Reinforcement Learning under Threats

In several reinforcement learning (RL) scenarios, mainly in security settings, there may be adversaries trying to interfere with the reward generating process. In this paper, we introduce Threatened Markov Decision Processes (TMDPs), which provide a framework to support a decision maker against a potential adversary in RL. Furthermore, we propose a level-$k$ thinking scheme resulting in a new learning framework to deal with TMDPs. After introducing our framework and deriving theoretical results, relevant empirical evidence is given via extensive experiments, showing the benefits of accounting for adversaries while the agent learns.Comment: Extends the verson published at the Proceedings of the AAAI Conference on Artificial Intelligence 33, https://www.aaai.org/ojs/index.php/AAAI/article/view/510

Gradual Drift Detection in Process Models Using Conformance Metrics

Changes, planned or unexpected, are common during the execution of real-life processes. Detecting these changes is a must for optimizing the performance of organizations running such processes. Most of the algorithms present in the state-of-the-art focus on the detection of sudden changes, leaving aside other types of changes. In this paper, we will focus on the automatic detection of gradual drifts, a special type of change, in which the cases of two models overlap during a period of time. The proposed algorithm relies on conformance checking metrics to carry out the automatic detection of the changes, performing also a fully automatic classification of these changes into sudden or gradual. The approach has been validated with a synthetic dataset consisting of 120 logs with different distributions of changes, getting better results in terms of detection and classification accuracy, delay and change region overlapping than the main state-of-the-art algorithms

Reproductive anatomy of Chondrichthyans: notes on specimen handling and sperm extraction. II. Sharks and chimaeras

[EN] Sperm extraction and artificial insemination may serve ex situ conservation initiatives for threatened sharks and related species. A comparison of the reproductive anatomy of eight chondrichthyans is presented in this study, emphasizing the important differences when performing these reproductive techniques. Additionally, we show how to obtain sperm samples from both living and dead specimens using cannulation, abdominal massage, or oviducal gland extraction. These tools can improve the success of breeding programs developed in aquaria and research facilities. The chondrichthyan fishes, which comprise sharks, rays, and chimaeras, are one of the most threatened groups of vertebrates on the planet. Given this situation, an additional strategy for the protection of these species could be the ex situ conservation projects developed in public aquaria and research centers. Nevertheless, to increase sustainability and to develop properly in situ reintroduction strategies, captive breeding techniques, such as sperm extraction and artificial insemination, should be developed. These techniques are commonly used in other threatened species and could be also used in chondrichthyans. However, the different reproductive morphologies found in this group can complicate both processes. Therefore, a comparison of the reproductive anatomy of eight distinct chondrichthyans, with an emphasis on those important differences when performing sperm extraction or artificial insemination, is carried out herein. Sharks and chimaeras belonging to the Scyliorhinidae, Carcharhinidae, Centrophoridae, Etmopteridae, Hexanchidae, and Chimaeridae families were obtained from commercial fisheries, public aquaria, and stranding events. In addition, the process of obtaining viable sperm samples through cannulation, abdominal massage, and oviducal gland extraction is described in detail for both living and dead animals.This research was partially funded by the Fundacion Biodiversidad (PRCV00683). P.G.-S. has a PhD contract from the European Union through the Operational Program of the European Social Fund (ESF) of the Comunitat Valenciana 2014-2020 ACIF 2018 (ACIF/2018/147). V.G. has a postdoc contract from the MICIU, Programa Juan de la Cierva-Incorporacion (IJCI-2017-34200).García-Salinas, P.; Gallego Albiach, V.; Asturiano, JF. (2021). Reproductive anatomy of Chondrichthyans: notes on specimen handling and sperm extraction. II. Sharks and chimaeras. Animals. 11(8):1-20. https://doi.org/10.3390/ani11082191S12011

Fondos cotizados : la inversión sectorial o indexada = Traded funds : the sectorial or indexed investment.

Este trabajo se centra en el debate de si es mejor inversión la de un índice de mercado o la de empresas de un mismo sector, tomando como vehículo de inversión los fondos de inversión cotizados o ETFs. Para contribuir a ello mediante datos reales, se selecciona una muestra de seis fondos, uno indexado al índice Standard and Poor’s americano y cinco de fondos dedicados a sectores diferentes que cotizan en este índice, todos de la misma entidad depositaria y con las mismas condiciones y ventajas. Tras un respaldo teórico de la Hipótesis de la Eficiencia, y del funcionamiento de dichos fondos, se realiza un análisis de la composición de los mismos y un análisis empírico de su rentabilidad y riesgo, llegando a la conclusión por sus resultados y proyecciones, que valorando las distintas posibilidades, la inversión en el propio índice ofrece mayores ventajas que la de un solo sector del mercado

Reproductive Anatomy of Chondrichthyans: Notes on Specimen Handling and Sperm Extraction. I. Rays and Skates

[EN] Many species of rays and skates are endangered, and ex situ conservation programs developed by research centers and public aquaria could improve this situation. To reproduce these species in captivity, scientists need to know how to extract their sperm and how to conduct their artificial insemination; however, the anatomical diversity of the reproductive organs of this group of animals is a handicap. A comparison of the reproductive anatomies of 11 distinct species is presented here, emphasizing the important differences between the species when performing sperm extraction or artificial insemination. In addition, the process of obtaining sperm samples from both live and dead males is described in detail, using both cannulation and abdominal massage. The superorder Batoidea (rays, skates, and relatives), constitutes one of the most threatened group of vertebrates. Strengthening ex situ conservation programs developed in research centers and public aquaria could be a way of addressing this situation. However, captive breeding programs must be improved to prevent the capture of wild animals and to develop proper in situ reintroduction strategies. Sperm extraction and artificial insemination are two techniques commonly used in other threatened species, which could also be used in rays and the like. However, the different reproductive morphologies present within this group of animals may hamper both processes. Here, we present a comparison of the reproductive anatomies of 11 distinct batoid species, emphasizing the important differences between the species when performing sperm extraction or artificial insemination. Both male and female animals, belonging to the Rajidae, Dasyatidae, Torpedinidae and Myliobatidae families, from the Mediterranean Sea were studied. In addition, we describe the procedure to extract sperm using both cannulation and abdominal massage, either from live or dead batoids Finally, the obtention of motile sperm recovered from the oviducal gland of females is described. These techniques generate a new range of possibilities for the conservation of these threatened species.This research was partially funded by the Fundacion Biodiversidad (PRCV00683). PGS has a PhD contract from the European Union through the Operational Program of the European Social Fund (ESF) of the Comunitat Valenciana 2014-2020 ACIF 2018 (ACIF/2018/147). VG has a postdoc contract from the MICIU, Programa Juan de la Cierva-Incorporacion (IJCI-2017-34200).García-Salinas, P.; Gallego Albiach, V.; Asturiano, JF. (2021). Reproductive Anatomy of Chondrichthyans: Notes on Specimen Handling and Sperm Extraction. I. Rays and Skates. Animals. 11(7):1-16. https://doi.org/10.3390/ani11071888S11611

Intracellular pH regulation and sperm motility in the European eel

[EN] Sperm activation involves ion fluxes as well as a previous maturation in the seminal plasma, something which has not been studied in depth in marine fish species. pH and potassium are probably involved in sperm maturation and motility in the European eel, as indicated in previous studies. In this work, the absolute intracellular concentration of potassium in European eel sperm has been determined for the first time. In addition, the intracellular pH (pH(i)) of quiescent eel spermatozoa was determined by two methods (nigericin and null point) that gave similar results, 7.4-7.6. The natural pH(i) range of sperm samples in the quiescent stage was 7.4-8.0, with no evident relationship with sperm motility. However, a linear correlation was seen between sperm motility and the pH of the diluent or extracellular pH (pH(e)), as well as between the pH(i) and the pH of the diluent. The change post-activation in seawater (ASW) depended on the initial pH(e) of the diluent medium. Activation with ASW induced an internal alkalinization of the cells when the sample had previously been diluted in a pH(e) 8, and no pH(i) variation when pH(e) was 8.0. These experiments indicated that a careful selection of the diluents should be performed before measuring natural pH(i) changes in sperm cells. Thus, studies on the specific seminal plasma composition of marine fish species are necessary before studying their physiology. Furthermore, our study indicates that intracellular alkalinization is not a universal fact during sperm activation. (C) 2020 Elsevier Inc. All rights reserved.Funded by the European Union's Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No 642893 (IMPRESS). Victor Gallego has a postdoc grant from the MICIU (Juan de la Cierva-Incorporacion; IJCI-2017-34200Pérez Igualada, LM.; Gallego Albiach, V.; Asturiano, JF. (2020). Intracellular pH regulation and sperm motility in the European eel. Theriogenology. 145:48-58. https://doi.org/10.1016/j.theriogenology.2020.01.026S4858145Nishigaki, T., José, O., González-Cota, A. L., Romero, F., Treviño, C. L., & Darszon, A. (2014). Intracellular pH in sperm physiology. Biochemical and Biophysical Research Communications, 450(3), 1149-1158. doi:10.1016/j.bbrc.2014.05.100Ohta, H., Ikeda, K., & Izawa, T. (1997). Increases in concentrations of potassium and bicarbonate ions promote acquisition of motility in vitro by Japanese eel spermatozoa. The Journal of Experimental Zoology, 277(2), 171-180. doi:10.1002/(sici)1097-010x(19970201)277:23.0.co;2-mOda, S., & Morisawa, M. (1993). Rises of intracellular Ca2+ and pH mediate the initiation of sperm motility by hyperosmolality in marine teleosts. Cell Motility and the Cytoskeleton, 25(2), 171-178. doi:10.1002/cm.970250206TANAKA, S., UTOH, T., YAMADA, Y., HORIE, N., OKAMURA, A., AKAZAWA, A., … KUROKURA, H. (2004). Role of sodium bicarbonate on the initiation of sperm motility in the Japanese eel. Fisheries Science, 70(5), 780-787. doi:10.1111/j.1444-2906.2004.00871.xGallego, V., Martínez-Pastor, F., Mazzeo, I., Peñaranda, D. S., Herráez, M. P., Asturiano, J. F., & Pérez, L. (2014). Intracellular changes in Ca2+, K+ and pH after sperm motility activation in the European eel (Anguilla anguilla): Preliminary results. Aquaculture, 418-419, 155-158. doi:10.1016/j.aquaculture.2013.10.022Vílchez, M. C., Morini, M., Peñaranda, D. S., Gallego, V., Asturiano, J. F., & Pérez, L. (2017). Role of potassium and pH on the initiation of sperm motility in the European eel. Comparative Biochemistry and Physiology Part A: Molecular & Integrative Physiology, 203, 210-219. doi:10.1016/j.cbpa.2016.09.024M�ri�n, T., Krasznai, Z., Balkay, L., Emri, M., & Tr�n, L. (1997). Role of extracellular and intracellular pH in carp sperm motility and modifications by hyperosmosis of regulation of the Na+/H+ exchanger. Cytometry, 27(4), 374-382. doi:10.1002/(sici)1097-0320(19970401)27:43.0.co;2-cPerez, L., Aturiano, J. F., Tomas, A., Zegrari, S., Barrera, R., Espinos, F. J., … Jover, M. (2000). Induction of maturation and spermiation in the male European eel: assessment of sperm quality throughout treatment. Journal of Fish Biology, 57(6), 1488-1504. doi:10.1111/j.1095-8649.2000.tb02227.xGallego, V., Carneiro, P. C. F., Mazzeo, I., Vílchez, M. C., Peñaranda, D. S., Soler, C., … Asturiano, J. F. (2013). Standardization of European eel (Anguilla anguilla) sperm motility evaluation by CASA software. Theriogenology, 79(7), 1034-1040. doi:10.1016/j.theriogenology.2013.01.019Mylonas, C. C., Duncan, N. J., & Asturiano, J. F. (2017). Hormonal manipulations for the enhancement of sperm production in cultured fish and evaluation of sperm quality. Aquaculture, 472, 21-44. doi:10.1016/j.aquaculture.2016.04.021Asturiano, J. F., Pérez, L., Garzón, D. L., Marco-Jiménez, F., Peñaranda, D. S., Vicente, J. S., & Jover, M. (2004). Physio-Chemical Characteristics of Seminal Plasma and Development of Media and Methods for the Cryopreservation of European eel Sperm. Fish Physiology and Biochemistry, 30(3-4), 283-293. doi:10.1007/s10695-005-1553-xThomas, J. A., Buchsbaum, R. N., Zimniak, A., & Racker, E. (1979). Intracellular pH measurements in Ehrlich ascites tumor cells utilizing spectroscopic probes generated in situ. Biochemistry, 18(11), 2210-2218. doi:10.1021/bi00578a012Krasznai, Z. (2003). Role of ion channels and membrane potential in the initiation of carp sperm motility. Aquatic Living Resources, 16(5), 445-449. doi:10.1016/s0990-7440(03)00054-8Miura, T., & Miura, C. I. (2003). Molecular control mechanisms of fish spermatogenesis. Fish Physiology and Biochemistry, 28(1-4), 181-186. doi:10.1023/b:fish.0000030522.71779.47Fechner, S., Alvarez, L., Bönigk, W., Müller, A., Berger, T. K., Pascal, R., … Kaupp, U. B. (2015). A K+-selective CNG channel orchestrates Ca2+ signalling in zebrafish sperm. eLife, 4. doi:10.7554/elife.07624Peñaranda, D., Pérez, L., Gallego, V., Barrera, R., Jover, M., & Asturiano, J. (2008). European Eel Sperm Diluent for Short-term Storage. Reproduction in Domestic Animals, 45(3), 407-415. doi:10.1111/j.1439-0531.2008.01206.xChauvaud, L., Cosson, J., Suquet, M., & Billard, R. (1995). Sperm motility in turbot, Scophthalmus marimus: initiation of movement and changes with time of swimming characteristics. Environmental Biology of Fishes, 43(4), 341-349. doi:10.1007/bf00001167Billard, R., Cosson, J., & Crim, L. W. (1993). Motility of fresh and aged halibut sperm. Aquatic Living Resources, 6(1), 67-75. doi:10.1051/alr:1993008Effer, B., Figueroa, E., Augsburger, A., & Valdebenito, I. (2013). Sperm biology of Merluccius australis: Sperm structure, semen characteristics and effects of pH, temperature and osmolality on sperm motility. Aquaculture, 408-409, 147-151. doi:10.1016/j.aquaculture.2013.05.040Inaba, K., Dréanno, C., & Cosson, J. (2003). Control of flatfish sperm motility by CO2and carbonic anhydrase. Cell Motility and the Cytoskeleton, 55(3), 174-187. doi:10.1002/cm.10119Lee, H. cheung, Johnson, C., & Epel, D. (1983). Changes in internal pH associated with initiation of motility and acrosome reaction of sea urchin sperm. Developmental Biology, 95(1), 31-45. doi:10.1016/0012-1606(83)90004-0Babcock, D. F., Rufo, G. A., & Lardy, H. A. (1983). Potassium-dependent increases in cytosolic pH stimulate metabolism and motility of mammalian sperm. Proceedings of the National Academy of Sciences, 80(5), 1327-1331. doi:10.1073/pnas.80.5.132